Adding in support for making any extension pipeline a "nightly" publishing pipeline#8740
Conversation
…Jurek. Yes _the_ Daniel Jurek.
So the important stuff:
- We know it's a "nightly" build because it's scheduled. That's it.
- We don't push GitHub releases for nightlies (it's a lot of clutter).
They only get pushed to our blob storage account.
- Nightlies have their own branch for the registry ('nightly') and their
own registry (cli/azd/extensions/registry.nightly.json).
So to enable this for an extension, just add a nightly publish schedule.
🔗 Linked Issue RequiredThanks for the contribution! Please link a GitHub issue to this PR by adding |
There was a problem hiding this comment.
Pull request overview
This PR adds “nightly” publishing support for azd extensions by treating scheduled Azure DevOps runs as nightly builds: publish artifacts to an always-latest storage folder, update a dedicated nightly registry on a separate nightly branch, and avoid creating GitHub Releases to prevent clutter.
Changes:
- Add scheduled-run (“nightly”) version suffixing (
-nightly.<buildId>/.nightly.<buildId>) and a registry normalization script for artifact URL rewriting. - Update extension pipeline templates to support nightly publishing semantics (storage-only publish + direct push to
nightlybranch registry). - Document the nightly registry and add Go tests that pin semver upgrade/promotion behavior for nightly versions.
Show a summary per file
| File | Description |
|---|---|
| eng/scripts/Update-NightlyExtensionRegistry.ps1 | Normalizes azd x publish --artifacts output by rewriting local artifact paths to the public storage URL and pruning to the latest nightly entry. |
| eng/scripts/Set-ExtensionVersionVariable.ps1 | Appends semver-valid nightly prerelease suffixes on scheduled runs using the pipeline build id. |
| eng/pipelines/templates/steps/publish-extension.yml | Adds a switch to skip GitHub release creation for nightlies and forces storage uploads to overwrite the nightly folder contents. |
| eng/pipelines/templates/stages/sign-extension.yml | Enables signing for scheduled (nightly) extension builds in addition to CI/manual builds. |
| eng/pipelines/templates/stages/release-azd-extension.yml | Extends release/sign stages to run for scheduled builds (nightlies) as well as manual builds. |
| eng/pipelines/templates/stages/publish-extension.yml | Implements nightly publish flow: storage-only publish and direct-push updates to cli/azd/extensions/registry.nightly.json on the nightly branch with retry-on-conflict. |
| cli/azd/pkg/extensions/nightly_versioning_test.go | Adds tests that pin semver ordering + upgrade/promotion behavior for nightly version formats. |
| cli/azd/docs/extensions/extension-resolution-and-versioning.md | Documents the nightly registry URL/source, retention model, and upgrade/promotion semantics. |
| cli/azd/docs/extensions/extension-framework.md | Adds a short “nightly registry” opt-in section pointing to the detailed versioning doc. |
Copilot's findings
- Files reviewed: 9/9 changed files
- Comments generated: 1
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
vhvb1989
left a comment
There was a problem hiding this comment.
LGTM, but wait for Daniel Jurek
|
@danieljurek ping |
JeffreyCA
left a comment
There was a problem hiding this comment.
Thanks Richard!
I think a good follow-up to consider is triggering on source changes only, considering we have quite a few azure.ai extensions, and not all are updated on a regular basis
Makes total sense - it's one of the checkboxes I was going to set since it's part of the scheduing apparatus, and (AFAIK) not in this yaml. |
|
/check-enforcer evaluate |
|
Checkenforcer appears stuck...actiavating manual override... |
|
/check-enforcer override |
Most of this comes from the discussion I had with Daniel Jurek. Yes the Daniel Jurek.
Basically:
To enable this for an extension, just add a nightly publish trigger in Azure DevOps (not as part of the YAML itself). I've tested this with a few builds, and even did an upgrade after the extension was pushed:
Here's the latest build of the azd-demo that produced a nightly build: dev.azure.com, publishing microsoft.azd.demo as a nightly build
First part of the work for #8729